2025-04-02 04:06:09,548 [ 801457 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse (runner:53, check_args_and_update_paths) 2025-04-02 04:06:09,548 [ 801457 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:97, check_args_and_update_paths) 2025-04-02 04:06:09,548 [ 801457 ] INFO : utils dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/utils (runner:108, check_args_and_update_paths) 2025-04-02 04:06:09,548 [ 801457 ] INFO : base_configs_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:110, check_args_and_update_paths) clickhouse_integration_tests_volume Running pytest container as: 'docker run --rm --name clickhouse_integration_tests_qmyh85 --privileged --dns-search='.' --memory=30709035008 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=5dc43a6382f0 -e DOCKER_BASE_TAG=8b2301119731 -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS=" -rfEps --run-id=1 --color=no --durations=0 test_reload_clusters_config/test.py::test_delete_cluster test_reload_clusters_config/test.py::test_update_one_cluster -vvv -ss" altinityinfra/integration-tests-runner:2165613c5fcd '. Start tests ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache Test order randomisation NOT enabled. Enable with --random-order or --random-order-bucket= rootdir: /ClickHouse/tests/integration configfile: pytest.ini plugins: timeout-2.3.1, repeat-0.9.3, order-1.0.0, reportlog-0.4.0, xdist-3.5.0, random-order-1.1.1 timeout: 900.0s timeout method: signal timeout func_only: False collecting ... collected 2 items test_reload_clusters_config/test.py::test_delete_cluster Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml PASSED test_reload_clusters_config/test.py::test_update_one_cluster FAILED =================================== FAILURES =================================== ___________________________ test_update_one_cluster ____________________________ started_cluster = def test_update_one_cluster(started_cluster): send_repeated_query("distributed") send_repeated_query("distributed2") > assert get_errors_count("test_cluster") > 0 E AssertionError: assert 0 > 0 E + where 0 = get_errors_count('test_cluster') test_reload_clusters_config/test.py:202: AssertionError ------------------------------ Captured log call ------------------------------- 2025-04-02 04:08:53 [ 656 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3647, query_and_get_error) 2025-04-02 04:09:05 [ 656 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3647, query_and_get_error) 2025-04-02 04:09:18 [ 656 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3647, query_and_get_error) 2025-04-02 04:09:30 [ 656 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3647, query_and_get_error) 2025-04-02 04:09:43 [ 656 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3647, query_and_get_error) 2025-04-02 04:09:55 [ 656 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3647, query_and_get_error) 2025-04-02 04:10:07 [ 656 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3647, query_and_get_error) 2025-04-02 04:10:19 [ 656 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3647, query_and_get_error) 2025-04-02 04:10:32 [ 656 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3647, query_and_get_error) 2025-04-02 04:10:45 [ 656 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3647, query_and_get_error) 2025-04-02 04:10:57 [ 656 ] DEBUG : Executing query SELECT errors_count FROM system.clusters WHERE cluster='test_cluster' and host_name='node_1' on node (cluster.py:3564, query) ---------------------------- Captured log teardown ----------------------------- 2025-04-02 04:10:57 [ 656 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_clusters_config/_instances-1/.env --project-name roottestreloadclustersconfig --file /ClickHouse/tests/integration/test_reload_clusters_config/_instances-1/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml stop --timeout 20] (cluster.py:120, run_and_check) 2025-04-02 04:11:09 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-node-1 Stopping (cluster.py:146, run_and_check) 2025-04-02 04:11:09 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-node-1 Stopped (cluster.py:146, run_and_check) 2025-04-02 04:11:09 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-zoo3-1 Stopping (cluster.py:146, run_and_check) 2025-04-02 04:11:09 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-zoo1-1 Stopping (cluster.py:146, run_and_check) 2025-04-02 04:11:09 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-zoo2-1 Stopping (cluster.py:146, run_and_check) 2025-04-02 04:11:09 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-zoo3-1 Stopped (cluster.py:146, run_and_check) 2025-04-02 04:11:09 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-zoo2-1 Stopped (cluster.py:146, run_and_check) 2025-04-02 04:11:09 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-zoo1-1 Stopped (cluster.py:146, run_and_check) 2025-04-02 04:11:09 [ 656 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_reload_clusters_config/_instances-1/node/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_reload_clusters_config/_instances-1/node/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:120, run_and_check) 2025-04-02 04:11:09 [ 656 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_clusters_config/_instances-1/.env --project-name roottestreloadclustersconfig --file /ClickHouse/tests/integration/test_reload_clusters_config/_instances-1/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml down --volumes] (cluster.py:120, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-node-1 Stopping (cluster.py:146, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-node-1 Stopped (cluster.py:146, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-node-1 Removing (cluster.py:146, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-node-1 Removed (cluster.py:146, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-zoo1-1 Stopping (cluster.py:146, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-zoo3-1 Stopping (cluster.py:146, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-zoo2-1 Stopping (cluster.py:146, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-zoo2-1 Stopped (cluster.py:146, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-zoo2-1 Removing (cluster.py:146, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-zoo1-1 Stopped (cluster.py:146, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-zoo1-1 Removing (cluster.py:146, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-zoo3-1 Stopped (cluster.py:146, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-zoo3-1 Removing (cluster.py:146, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-zoo3-1 Removed (cluster.py:146, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-zoo1-1 Removed (cluster.py:146, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stderr: Container roottestreloadclustersconfig-zoo2-1 Removed (cluster.py:146, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stderr: Network roottestreloadclustersconfig_default Removing (cluster.py:146, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stderr: Network roottestreloadclustersconfig_default Removed (cluster.py:146, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2025-04-02 04:11:10 [ 656 ] DEBUG : Docker networks for project roottestreloadclustersconfig are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2025-04-02 04:11:10 [ 656 ] DEBUG : Docker containers for project roottestreloadclustersconfig are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2025-04-02 04:11:10 [ 656 ] DEBUG : Docker volumes for project roottestreloadclustersconfig are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2025-04-02 04:11:10 [ 656 ] DEBUG : Command:[docker container list --all --filter name='^/roottestreloadclustersconfig-.*-1$' --format '{{.ID}}:{{.Names}}'] (cluster.py:120, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2025-04-02 04:11:10 [ 656 ] DEBUG : No running containers for project: roottestreloadclustersconfig (cluster.py:904, cleanup) 2025-04-02 04:11:10 [ 656 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2025-04-02 04:11:10 [ 656 ] DEBUG : Trying to prune unused images... (cluster.py:926, cleanup) 2025-04-02 04:11:10 [ 656 ] DEBUG : Command:[docker image prune -f] (cluster.py:120, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:144, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Images pruned (cluster.py:929, cleanup) 2025-04-02 04:11:10 [ 656 ] DEBUG : Trying to prune unused volumes... (cluster.py:935, cleanup) 2025-04-02 04:11:10 [ 656 ] DEBUG : Command:[docker volume ls | wc -l] (cluster.py:120, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Stdout:1 (cluster.py:144, run_and_check) 2025-04-02 04:11:10 [ 656 ] DEBUG : Volumes pruned: 1 (cluster.py:940, cleanup) ============================== slowest durations =============================== 135.69s call test_reload_clusters_config/test.py::test_delete_cluster 123.93s call test_reload_clusters_config/test.py::test_update_one_cluster 24.03s setup test_reload_clusters_config/test.py::test_delete_cluster 12.94s teardown test_reload_clusters_config/test.py::test_update_one_cluster 0.00s teardown test_reload_clusters_config/test.py::test_delete_cluster 0.00s setup test_reload_clusters_config/test.py::test_update_one_cluster =========================== short test summary info ============================ FAILED test_reload_clusters_config/test.py::test_update_one_cluster - Asserti... PASSED test_reload_clusters_config/test.py::test_delete_cluster =================== 1 failed, 1 passed in 296.69s (0:04:56) ==================== Traceback (most recent call last): File "/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration/./runner", line 528, in subprocess.check_call(cmd, shell=True, bufsize=0) File "/usr/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'docker run --rm --name clickhouse_integration_tests_qmyh85 --privileged --dns-search='.' --memory=30709035008 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=5dc43a6382f0 -e DOCKER_BASE_TAG=8b2301119731 -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS=" -rfEps --run-id=1 --color=no --durations=0 test_reload_clusters_config/test.py::test_delete_cluster test_reload_clusters_config/test.py::test_update_one_cluster -vvv -ss" altinityinfra/integration-tests-runner:2165613c5fcd ' returned non-zero exit status 1.